A Many-facet Rasch Model to Detect Halo Effect in Three Types of Raters

نویسنده

  • Farahman Farrokhi
چکیده

Raters play a central role in rater-mediated assessment, and rater variability manifested in various forms including rater errors contributes to construct-irrelevant variance which can adversely affect an examinee’s test score. Halo effect as a subcomponent of rater errors is one of the most pervasive errors which, if not detected, can result in obscuring an examinee’s score and threatening validity and fairness of second language performance assessment. To that end, the present study is an endeavor to detect halo effect in L2 essays, using a relatively newly employed methodology, a many-facet Rasch model (MFRM) in language assessment. The participants in this study consisted of 194 raters—subdivided into self-rater, peer-rater, and teacher rater—who rated 188 essays written by 188 undergraduate Iranian English majors at two state-run universities in Iran. The collected data were rated using a 6-point analytic rating scale and were analyzed using the latest version of Facets 3.68.0 to answer the research question of the study. The results of facets analysis showed that, at group level, the raters did not exhibit any sign of halo effect, but, at individual level, all rater types displayed considerable halo effect. Further analysis revealed that rater types were unanimous about halo effect on four items and that self-rater showed more of a halo effect compared to the other two rater types.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Rater Errors among Peer-Assessors: Applying the Many-Facet Rasch Measurement Model

In this study, the researcher used the many-facet Rasch measurement model (MFRM) to detect two pervasive rater errors among peer-assessors rating EFL essays. The researcher also compared the ratings of peer-assessors to those of teacher assessors to gain a clearer understanding of the ratings of peer-assessors. To that end, the researcher used a fully crossed design in which all peer-assessors ...

متن کامل

Analysis of rater severity on written expression exam using Many Faceted Rasch Measurement

This paper describes how a Many Faceted Rasch Measurement (MFRM) approach can be applied to performance assessment focusing on rater analysis. The article provides an introduction to MFRM, a description of MFRM analysis procedures, and an example to illustrate how to examine the effects of various sources of variability on test takers’ performance on a writing test by means of a MFRM analysis. ...

متن کامل

Many-Facet Rasch Measurement

This chapter provides an introductory overview of many-facet Rasch measurement (MFRM). Broadly speaking, MFRM refers to a class of measurement models that extend the basic Rasch model by incorporating more variables (or facets) than the two that are typically included in a test (i.e., examinees and items), such as raters, scoring criteria, and tasks. Throughout the chapter, a sample of rating d...

متن کامل

Detecting and measuring rater effects using many-facet Rasch measurement: part I.

The purpose of this two-part paper is to introduce researchers to the many-facet Rasch measurement (MFRM) approach for detecting and measuring rater effects. The researcher will learn how to use the Facets (Linacre, 2001) computer program to study five effects: leniency/severity, central tendency, randomness, halo, and differential leniency/severity. Part 1 of the paper provides critical backgr...

متن کامل

Language Complexity, Accuracy and Fluency in Different Types of Writing Paragraph: Do the Raters Notice Such Effect

The aim of the present study was to investigate the effects of two types of paragraph on EFL learners’ written production. It addressed the issue of how three aspects of language production (i.e. complexity, accuracy, and fluency) vary among two types of paragraphs (i.e. paragraphs of chronology and cause-effect) written by EFL learners. Thirty intermediate level learners of English participate...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2011